3 research outputs found
Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer
In recent years, privacy-preserving methods for deep learning have become an
urgent problem. Accordingly, we propose the combined use of federated learning
(FL) and encrypted images for privacy-preserving image classification under the
use of the vision transformer (ViT). The proposed method allows us not only to
train models over multiple participants without directly sharing their raw data
but to also protect the privacy of test (query) images for the first time. In
addition, it can also maintain the same accuracy as normally trained models. In
an experiment, the proposed method was demonstrated to well work without any
performance degradation on the CIFAR-10 and CIFAR-100 datasets
Block-Wise Encryption for Reliable Vision Transformer models
This article presents block-wise image encryption for the vision transformer
and its applications. Perceptual image encryption for deep learning enables us
not only to protect the visual information of plain images but to also embed
unique features controlled with a key into images and models. However, when
using conventional perceptual encryption methods, the performance of models is
degraded due to the influence of encryption. In this paper, we focus on
block-wise encryption for the vision transformer, and we introduce three
applications: privacy-preserving image classification, access control, and the
combined use of federated learning and encrypted images. Our scheme can have
the same performance as models without any encryption, and it does not require
any network modification. It also allows us to easily update the secret key. In
experiments, the effectiveness of the scheme is demonstrated in terms of
performance degradation and access control on the CIFAR10 and CIFAR-100
datasets.Comment: 7 figures, 3 tables. arXiv admin note: substantial text overlap with
arXiv:2207.0536
Privacy-Preserving Semantic Segmentation Using Vision Transformer
In this paper, we propose a privacy-preserving semantic segmentation method that uses encrypted images and models with the vision transformer (ViT), called the segmentation transformer (SETR). The combined use of encrypted images and SETR allows us not only to apply images without sensitive visual information to SETR as query images but to also maintain the same accuracy as that of using plain images. Previously, privacy-preserving methods with encrypted images for deep neural networks have focused on image classification tasks. In addition, the conventional methods result in a lower accuracy than models trained with plain images due to the influence of image encryption. To overcome these issues, a novel method for privacy-preserving semantic segmentation is proposed by using an embedding that the ViT structure has for the first time. In experiments, the proposed privacy-preserving semantic segmentation was demonstrated to have the same accuracy as that of using plain images under the use of encrypted images